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Occasional Brief 

Does Maine’s Middle School Laptop Program Improve Learning? 

David L. Silvernail University of Southern Maine 

“Laptop Students Still Test The Same.” This headline from an August 
2004 Portland Press Herald newspaper article caused a great deal of angst on 
the part of proponents of Maine’s innovative middle school laptop program, 
while it fueled the fires for the critics of the program. The newspaper article 
states in part: 

“The first children to use laptops in the class show no improvement in statewide 
ME A tests after two years. . .The eighth grade scores for reading, writing, math, 
and science in the Maine Education Assessments were essentially unchanged in 
the past school year compared with the previous two years.” (Bell, August 
10, 2004, Al). 

It is true that overall performance on the 8 th grade Maine Education 
Assessments (MEAs) has not changed appreciably in the last three years. And 
at first blush this may be somewhat disconcerting, particularly in light of the 
fact that other laptop programs appear to have had positive impacts on student 
achievement. For example, several studies report improved writing scores. 
Rockman et al. (2000) found that students who used laptops outperformed 
students who did not use them, in four specific areas of writing, Jeroski (2003) 
reported that giving students laptops increased the percentage of students who 
met performance standards by 22% over the course of one year, and Lowther 
(2003) found significant improvements in writing and problem-solving 
performance for students given 24 hour access to laptops. Mann et al (1999) 
reported higher performance on a state examination for students with laptops, 
and in one South Carolina middle school, students with laptops sustained their 
achievement gains over their middle school years, in contrast to students who 
did not have laptops (Stevenson, 1998). 

So there is a growing body of research showing the academic benefits of 
laptops. But even given this evidence from other states, Maine’s MEA 
performance still is not surprising for several reasons. 
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Given the manner in which the laptop program was implemented, 
changes in test scores on any statewide standardized tests should not be 
expected. Essentially, implementation of the program was left in the hands of 
each school district. The State did not dictate when and how the program was 
to be implemented in the 240+ Maine middle schools, nor did the State say how 
the laptops were to be used inside these schools. As a result, some schools 
began implementing the program right away, some several months later. Some 
used the laptops in just selected classes and in a limited number of disciplines, 
others in all classes. Some schools gave teachers considerable help and 
support in learning to use the laptops, while others did not. And some schools 
let the students take the laptop home, others did not. Consequently, the 
variations were so great it is unrealistic to expect clear, consistent changes on 
any standardized test. Even within specific schools and classrooms, wide 
variations in the way the laptop program was implemented would, in all 
likelihood, have differing effects on any achievement test scores. To determine 
if, and to what extent, standardized achievement test scores would improve, 
one needs a much more controlled implementation strategy. 

Second, even if the implementation strategy had been more uniform, 
Maine’s assessment program is not designed to measure growth from year to 
year for a particular group of students. The MEAs merely provide a snapshot of 
the performance of a single 8 th grade group at a particular point in time. It is a 
one-time assessment given to all 8 th graders every year, and different groups of 
8 th graders each year. In other words, different groups of 8 th graders are tested 
each year, not the same groups from year to year. Thus, the 8 th grade MEAs 
cannot be used to assess the growth in learning in a specific group of 7 th and 
8 th graders over the course of two years. 

Third, and possibly most importantly, the present MEA is not designed to 
measure the type of learning one might expect as a result of the introduction of 
an ubiquitous 1-to-l laptop program. The goal of the Maine Learning 
Technology Initiative, the initiative that has spearheaded Maine’s laptop 



2 




program, was to prepare Maine’s students for the 21 st Century. In the words of 
the original task force: 

We live in a world that is increasingly complex and where change is 
increasingly rampant. Driving much of this complexity and change are new 
concepts and a new economy based on powerful, ubiquitous computer technology 
linked to the internet. 

Our schools are challenged to prepare young people to navigate and 
prosper in this world, with technology as an ally rather than an obstacle. The 
challenge is familiar, but the imperative is new: we must prepare young people to 
thrive in a world that doesn 7 exist yet, to grapple with problems and construct 
new knowledge which is barely visible to us today. It is no longer adequate to 
prepare some of our young people to high levels of learning and technological 
literacy; we must prepare all for the demands of a world in which workers and 
citizens will be required to use and create knowledge, and embrace technology as 
a powerful tool to do so. 

If technology is a challenge for our educational system, it is also part of 
the solution. To move all students to high levels of learning and technological 
literacy, all students will need access to technology when and where it can be 
most effectively incorporated into learning. (Task Force on Maine’s Learning 
Technology Endowment, 2001, p.i.). 

Thus, a major goal of Maine’s laptop program has been to help students 
acquire 21 st Century skills using technology. However, the current MEA is 
designed to assess, at best, gateway skills and basic knowledge; skills and 
knowledge which may be necessary, but not sufficient for the demands of the 
21 st century. Furthermore, the MEAs are designed to determine if students 
have acquired knowledge, rather than created new knowledge; a skill which is 
becoming increasingly important as we move into the new century. 

Consider these science examples. Below is an item that, according to the 
MEA test developers, is designed to assess students’ ability to communicate 
effectively in the application of science and technology: 



3 




Boiling 


; Points of Carbon < 


Compounds 


Name 


Number of 
Carbon Atoms 


Boiling Point 


Ethane 


2 


-88°C 


Butane 


4 


o°c 


Hexane 


6 


69°C 


Octane 


8 


126°C 


Decane 


10 


174°C 



2. Which of the following is the best estimate of the boiling point 
of heptane, a compound that contains seven carbon atoms? 

A. -40°C C. 95°C 

B. 35°C D. 150°C 



How does this compare to 21 st Century communication skills? The Partnership 
for 21 st Century Skills, a partnership of educators, businesses and community 
leaders, and parents, describes a 21 st Century skill of communicating science 
information as follows: 



• Use appropriate tools to analyze and synthesize information (e.g., diagrams, flow 
charts, frequency tables, bar graphs, line graphs, and stem-and-leaf plots) to draw 
conclusions and implications based on investigations of an issue or question. 

For example, compile qualitative and quantitative data gathered through an 
investigation of past and current earthquake epicenters and regions of volcanic 
activity in order to identify needs and problems arising from events relating to the 
earth’s crust. 



Consider another MEA science item, one designed according to the test 
developers to have students apply inquiry and problem-solving approaches 
in science and technology : 
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] 


Minerals With A 


Metallic Luster 


Mineral 


Color 


Streak 


Hardness 


Specific 

Gravity 


graphite (C) 


steel gray 


black 


1 


2 


galena (PbS) 


silver gray 


gray 


2.5 


7.5 


chalcopyrite (CuFeS 2 ) 


golden yellow 


greenish black 


4 


4.3 


pyrite (FeS 2 ) 


brass yellow 


greenish black 


6 


5 


hematite (Fe 2 C> 3 ) 


steel gray 


reddish brown 


6 


5.2 


magnetite (FesCb) 


black 


black 


6 


5.2 



3. Which characteristic would be most useful in distinguishing a sample of 
hematite from the other minerals listed in the table? 



A. color C. hardness 

B. streak D. specific gravity 



And here is a 21 st Century problem solving skill : 



• Collect real-time observations and data synthesizing and building upon existing 
infonnation (e.g., online databases, NOAA, EPA, USGS) to solve problems. 

For example, collect data and search print and electronic resources to 
gather and record past data on the change in the turbidity of a river after a 
rainfall and its effect on the plants and animals living in this habitat. 



It is clear from these examples that what is being asked of students on 
the MEA, and the 21 st Century skills we are now asking students to 
demonstrate in classrooms all across the state, are very different. The MEA 
simply does not measure the kinds of knowledge and skills we should expect 
Maine’s middle school students to acquire and create through the use of 
laptops. (The one exception may be in the area of writing, which is discussed in 
more detail later). The MEA items only require students to recall information, 
and to select from a list of multiple choice answers. Twenty-first Century skills, 
on-the-other-hand, require students to locate information, sort it, analyze it, 
formulate and test solutions, and communicate it effectively to others. As 
Rockman (2003) says, “Computers don’t provide content, they offer the tools to 
access, manipulate, and organize content.” So it is not surprising that the 
introduction of the laptops in Maine’s middle schools has not translated directly 
into higher scores on the statewide MEA achievement tests. 
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Is there evidence that learning has changed and improved as a result of 
the laptops performance in Maine? Unfortunately, to date we do not have 
exhaustive evidence of how, and how much, the introduction of laptops has 
changed learning. But we do have what British physicist Eddington (1929) 
called “pointer readings” - evidence that begins to answer questions and points 
toward particular conclusions. In this case, we have evidence of pointer 
readings of several different types, that when taken together may point toward 
a particular conclusion. 

First, there is evidence that teachers all across the state are developing 
lessons and curriculum-embedded assessments designed to help their students 
acquire 21 st Century skills like those describe above. For example, here are 
some sample tasks teachers are assigning their students: 

1 . To leam about the biological cell students used a website to compare a cell’s 
organelles and functions to those of a city. Then students designed and made 
physical modules of human organizations to compare to a cell. 

2. To learn about calculating volume and density of irregularly shaped objects, students 
collected experimental data, graphed it using their laptops, created formulas for 
doing the calculations, and modeled and tested the formulas on new sets of objects. 

3. To examine discrete mathematics concepts, students measured their height, arm span 
and foot size, used their laptops to collect class data, calculated averages, developed 
predictions, and then tested their predictions with a second data collection cycle 
some months later. 

4. To understand personal and social responses to natural disasters, such as the tsunami, 
students searched online databases, sorted and synthesized this information in their 
Social Studies classes, used websites to examine the physics of the tsunami in their 
Science classes, and developed skills of oral and written slideshow presentations in 
their Language Arts classes. 

Second, we have evidence from teachers’ assessments of the impacts of 
the laptops on their students and their learning. As part of the first year 
evaluation of Maine’s middle school laptop program, teachers were asked about 
the impact of the laptops on their student’s learning. These teacher 
assessments may be seen in Table 1 . As indicated by this evidence, teachers 
believe that the laptops have improved the quality of learning, and improved 
students understanding of what they are learning (Silvernail and Lane 2004). 
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Table 1 

Teacher Assessments 



100 % 
90% 
80% - 
70% - 
60% 
50% 
40% H 
30% 
20 % 
10% H 
o% 



Students in my 
classroom are 
more actively 
involved in their 
own learning 
when we use the 
laptops. 



My students are 
more engaged 
when we are using 
the laptops. 



Students in my Laptops allow my The quality of my My students are 
classroom do more students to get students' work better able to 



work when they 
are using their 
laptops. 



their work done 
more quickly. 



increases when we 
use laptops. 



understand when 
they use the 
laptops. 



And this evidence should not be taken lightly. Research has shown that 
teachers’ judgments are very accurate measures of actual student achievement 
(Coladarci and Hodge, 1988; Chakuluk and Kenny, 1993; and Fuller, 2000). 

As Guskey (1996) asserts: 

“ Because teachers know their students, understand various dimensions of 
students’ work, and have clear notions of the progress made, their subjective 
perceptions may yield very accurate descriptions of what students have learned 

Third, we have some evidence from parents, another group of individuals 
adept at assessing changes in children’s learning. For example, a survey of 
middle school parents in one Maine school district yielded some very interesting 
results (Palfrey, 2004). As shown in Table 2 on the next page, over 80% of the 
parents that responded to the survey indicated that the laptops have positively 
impacted their child’s learning, and almost 60% indicated laptops have improved 
their child’s organizational skills, skills specifically desired by colleges and 
employers alike. 
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Table 2 

Middle School Parent Assessments 



Item 


Strongly 

Disagree 


Disagree 


Neutral 


Agree 


Strongly 

Agree 


Too Early 
To Tell 


1 . Laptops have positively impacted 
how my son or daughter learns at 
school. 


NG 


3.2% 


10.2% 


29.3% 


51.5% 


5.7% 


2. My son or daughter is more 
organized as a result of using a 
laptop. 


1.2% 


10.1% 


22.6% 


29.6% 


28.9% 


7.5% 



As one parent put it: 

“My son was in the first 7 th grade class to have laptops. The first year, his grades 
improved dramatically and continued to do so every year since. / attribute this 
mostly to the ability to access information more easily whether it be on the 
internet or in his files. Organization has definitely improved and interest in 
school projects heightened due to having access to a variety of resources.” 

Another source of evidence is a preliminary analysis of differences in 

performance of students completing the writing section of the MEA, some using 

paper and pencil and some using their laptops. As noted earlier, the one area 

where the introduction of laptops may be discernable in MEA results is in the 

area of writing. In this case, the MEA does measure directly a skill area which is 

not only important today, but also one that will continue to be important in the 

21 st Century - that is, the skill of communicating ideas effectively in written form. 

In 2004, approximately 10% of the 8 th graders completed the writing section 

of the MEA online using laptops, while the remaining 90% completed the writing 

section the traditional way - using paper and pencil. Table 3, on the next page, 

reports the results from an analysis of these writing scores by the Maine 

Department of Education (MDOE, 2004], The analysis revealed that the students 

who completed the assessment online scored on average 3 points higher than 

those who wrote their responses on paper, a difference greater than the standard 

error of measurement and approximately equal to 'A of a standard deviation better 

than the paper and pencil scores. And the difference was even greater for higher 

performing students. Obviously this is not definitive evidence, and it is unclear 

whether the results were because laptops had significantly improved students 
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Table 3 

Comparison of Online and Paper ME A Writing Sample 



Statistics 


Wri 

Online 


ting 

Paper 


Average 


539.6 


536.6 


SD 


9.9 


9.2 


n 


1,816 


14,870 



written communication skills or whether the sudden shift back to providing a 
writing sample using paper and pencil had hindered the performance of some 
students. But the evidence does provide one more pointer reading indicating 
laptops have positively affected students’ academic performance in some way. 

Another source of evidence is that collected by individual teachers with 
their students. One example comes from a middle school mathematics teacher. 
Table 4 reports this teacher’s analysis of student performance in his Basic 
Algebra classes before and after the introduction of laptops (Toy, 2004). As the 

Table 4 



Student Performance Before and After Introduction of Laptops 



2001-02 

Year Before Laptops 


Quizzes 

(No changes made) 


2002-03 

1 st Year of Laptops 


Increase in Class 
Average 


74 


Decimal/Fractions 


91 


17 


71 


Mixed Numbers 


84 


13 


79 


Decimals 


89 


10 


87 


Statistics 


96 


9 


85 


Fractions 


94 


9 


88 


Patterns 


96 


8 


81 


Systems of Equations 


89 


8 


j 82 


Negatives 


88 


6 


85 


Percents 


90 


5 


86 


Linear Equations 


91 


5 


94 


Coordinates 


98 


4 


82 


Geometry 


85 


3 



data shows, performance improved anywhere from 3% to 17%, with an average 
improvement of slightly more than 8%. This is only one example but it 
demonstrates that improvements in achievement may be seen if changes are 
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assessed at the individual school or class level using consistent, uniform 
curriculum-embedded assessments. 

Finally, Maine very soon will have another “pointer reading” on the 
impact of the laptop program on student learning. Maine was awarded one of 
only ten federally funded competitive grants designed to examine the impact of 
technology on classrooms and learning. This three-year grant is providing 
Maine an opportunity to examine up-close, and in a very systematic and 
structured way, the impact of laptops on middle school students’ mathematics 
learning. Although it is too early to have conclusive results (i.e. , at the 
midpoint of the three year study), early reports from teachers are very positive. 
Teachers report that with the use of the laptops and online resources in their 
classrooms students are learning facts, rule making, and concepts quicker, 
developing deeper understandings of abstract mathematics processes, and 
learning how to apply mathematics in solving real-world problems. And in 
Spring 2005 Maine will have completed the first phase of the study and will 
have test results for over 4500 middle school students, evidence collected from 
a rigorously designed experimental study of Maine’s laptop program. 

In summary, then, it appears that Maine’s introduction of its ubiquitous 
1 -to- 1 laptop program has been beneficial to learning, if one looks in the right 
places. There is growing positive evidence from teachers, parents, classroom 
assignments and tests, and even in one case, on a standardized test, all 
pointing toward a similar conclusion. Students are learning more, better, and 
quicker with the laptops. The collection of additional evidence needs to 
continue in order to determine how widespread the impacts are for learning, 
and to determine the accumulative effects of this new level of learning on high 
school student performances. But these “pointer readings” are very promising, 
and bode well for the future achievement of Maine’s middle school students. 
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